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LIKELIHOOD  RATIO  TEST  FOR  THE  EQUIVALENCE  OF  TWO 
AUTOREGRESSIVE  MOVING- AVERAGE  TIME  SERIES 


1.  INTRODUCTION 


In  many  areas  of  spatial  and  temporal  signal  processing,  the  analysis  of  the  relationship 
between  two  (or  more)  time  series  is  of  interest;  time  delay  estimation  is  one  example. 
Crosscorrelation  analysis  can  provide  insight  into  a  linear  relationship  between  the 
signals;  however,  it  does  not  compare  their  structural  details.  For  future  sonar  systems, 
the  procedures  described  in  this  report  have  the  potential  to  improve  sonar  operator 
performance  by  automatically  determining  if  two  signals  or  sonar  tracks  are  associated  with 
the  same  source. 

Specifically,  two  independent  autoregressive  moving-average  (ARMA)  time  series  are 
analyzed  to  determine  whether  or  not  the  series  may  be  characterized  by  the  same  ARMA 
model.  If  the  same  model  can  be  successfully  applied  to  each  series,  the  implication  is  that 
the  series  may  have  the  same  parent  process;  further  analysis  of  the  series  is  then  warranted. 

To  test  the  equivalence  of  two  ARMA  time  series  models,  a  likelihood  ratio  statistic 
is  derived.  The  test  statistic  —2  •  log  T,  where  T  is  the  likelihood  ratio  statistic,  has  an 
asymptotic  chi-square  distribution  with  r  degrees  of  freedom.  This  value  is  defined  as  the 
number  of  model  parameters  in  series  1  that  are  being  tested  for  equality  to  the  same  model 
parameters  in  series  2. 

In  section  2,  the  likelihood  ratio  test  is  derived  to  measure  the  equivalence  of  two 
autoregressive  (AR)  processes  with  Gaussian  innovations.  Because  the  AR  model 
parameters  are  easier  to  estimate  and  because  many  time  series  can  be  characterized  as  AR 
processes,  this  special  case  is  treated  separately. 

Section  3  presents  the  likelihood  ratio  for  the  equivalence  of  two  ARMA  processes  with 
Gaussian  innovations.  Results  from  simulated  data  are  described  in  section  4.  Section  5 
discusses  a  general  form  of  the  test,  as  well  as  provides  the  conclusions. 


1/(2  blank) 


2.  DERIVATION  OF  THE  LIKELIHOOD  RATIO  FOR  AR  PROCESSES 


Let  X\,  X2,  ...  ,  xnx  and  Vi,  U2>  —  ,  Vny  be  realizations  of  the  autoregressions  with 
respective  means  px  and  py  defined  by 


px 

Xt  =  Px  +  ^2  Oik  {xt-k  —  Px)  +  Ut 

k  —  1 

and 

py 

yt  =  v-y  +  ^Ph  ( yt-k  -  t*y)  +  vt ,  (i) 

jt=i 


where  a.\,  a 2,  ...  ,  apx  and  /3lt  P2,  ...  ,  p^  are  the  parameters  for  the  AR  pro¬ 
cesses  xt  and  yt  with  model  orders  px  and  py ,  respectively.  The  values  ut  and  vt  are 
independent  Gaussian  random  variables  with 


E[ut]  =  px  , 

E[vt]  =  pt  > 

E[usut]  -  a l6{s-t), 
E[vsvt]  =  a\  6(s  - 1 )  , 
E[usvt]  —  0  for  all  s  and  t  , 


where  £?[•]  denotes  the  statistical  expectation  operator  and  S(-)  is  the  Kronecker  delta 
function. 

For  the  present  discussion,  it  is  of  interest  to  know  whether  xt  (t  =  1,  2,  ...  ,  nx)  and 
yt  ( t  =  1,  2,  ...  ,  ny)  are  realizations  of  the  same  AR  process.  Since  the  objective  is  to  test 
whether  xt  and  y t  have  the  same  parent  AR  process,  px  and  py  are  set  equal  to  p. 

Because  ut  and  vt  are  assumed  to  be  independent  Gaussian  random  variables,  the 
likelihood  function  of  and  conditioned  on  and  {yt}\  is 


L{x,  y;  a,  (3,  oj,  cr*,  /zx,  py)  =  L(x;  a,  px)  *  L(  y;  (3,  al,  py)  , 


(2) 
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where 


L(x  ;  a,  cr2,  px)  =  (2tt<x2)  <“*  P)/2exp(-  (*t  -  A*»  “  E  «*(**-*  “  Vx)?  fal) 


£(y ;  P,cl,  v-y) 


(27Tt72)  (n2/  p)/2exp(-  J2(yt-Vy- it  Pk{yt-k  -  py))2/2a2v)  . 

t=p+ 1  fc=1 


The  vectors  x,  y,  a,  and  /3  are  defined  by 


X  —  (Xjj+i,  ^m)  > 

y  =  {iJp+li  Vp+2i  2/ny)  > 

Ol  (oil,  0J2;  Oip)  j 


/3  =  (0i,/32,-,0P)T, 

where  the  superscript  T  denotes  the  transpose  of  a  matrix. 

The  parameters  examined  and  the  form  of  the  four  likelihood  ratio  tests  are  as  follows: 

1.  A  test  for  equal  AR  parameters  (a*.  =  /3k,  k  =  1,  2, ...  ,  p)  and  innovations  variances 
(<*-<*): 

max  L(x,y  ;  a,/3,<72,cT2,  /za,  /xy) 

^  a  =  0  ,  a2u  =  a2  ,  px  ,  py 

•L  —  T  /  /o22  \  ' 

max  T(x,y  ;  at,  0^,0%,  px,  py) 

a,  (3,  crl,  a2v,  px,  py 

2.  A  test  for  equal  AR  parameters  (a*  =  (3k,  k  =  1,  2,  ...  ,  p),  innovations  variances 
(cr2  =  cr2) ,  and  process  means  (px  =  py) : 

max  L(x,  y;  a,  >3,  <72,  <r2,  px,  py) 

/v  —  /3  *  1  —  *  1  ^-2  —  j—2 


<*  =  0,Px  =  /y  gj  =  fj _ 

max  L(x,  y;  a,  /3,  <r2,  a2,/^,  /^) 

ft)  G U1  (Tyj  pxj  Py 
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3.  A  test  for  equal  AR  parameters  (ak  =  fik,  k  =  1,  2,  ...  ,  p): 


max  L(x,y,oc,P,al,(r2v,  nx,  fj,y) 

T  ....  a  =  /3,  _ 

maxL(x,y;a,/3,olto*) 

<*,  A  0-u.  Py 

4.  A  test  for  equal  AR  parameters  (o^  =  (3k  ,  k  —  1,  2,  ...  ,  p  )  and  process  means 

(Mx  =  Hy): 

max  L(x, y;  a,  (3,<j2u,  a2v, px,  py) 

T._  Q  =  A  Mr  =  /*y _ 

max  L(x,  y;  a,  P,a\ ,  a\,  \ix,  /xy) 

a,  A  a*,  a\ ,  px, 

To  evaluate  T,  a  value  of  the  model  order  p  must  be  provided.  Using  the  Akaike 
information  criteria  (AIC)  or  one  of  the  other  selection  procedures  available1-3  allows 
model  orders  px ,  py,  and  pxy  to  be  obtained  for  xt,  yt)  and  the  pooled  data  of  xt  and  yt 
respectively.  Three  possible  ways  to  select  p  are 

(a)  p  =  minimum  of  px  and  py, 

(b)  p  =  maximum  of  px  and  py,  or 

(c)  p  =  pxy. 


Simulation  experiments  have  shown  that  selecting  p  using  (a)  or  (b)  results  in  a  liberal 
test  that  rejects  the  null  hypothesis  when  it  is  true  more  often  than  is  predicted  by  the 
probability  distribution  of  the  test  statistic.  Using  (c),  or  p  —  pxy  (the  selected  model  order 
of  the  pooled  data),  yields  a  test  statistic  that  has  a  good  fit  to  the  chi-square  distribution 
with  the  appropriate  degrees  of  freedom. 


2.1  TEST  FOR  EQUAL  AR  PARAMETERS  AND 
INNOVATIONS  VARIANCES 

This  section  examines  whether  ak  =  fik  ( k  =  1,  2,  ...  ,  p)  and  cr2x  =  a2.  Since  this 
test  is  not  concerned  with  the  means  px  and  py,  it  is  assumed  that  the  sample  means 
x  =  (1/nr)  xt  and  V  =  (1/rcp)  ]C”=i  Y*  have  been  subtracted  from  the  realizations 
xt  (where  t  =  1,  2,  ...  ,  nx )  and  yt  (where  t  =  1,  2,  ...  ,ny),  respectively. 

The  likelihood  ratio  test  statistic  to  determine  whether  the  two  realizations  can  be 
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characterized  by  the  same  AR  process  is  given  by  equation  (5).  The  maximum  of  the 
denominator  is  obtained  at  the  values  of  a  and  (5  that  satisfy 


Aa  =  a 


and 

B/3  =  b  , 


(9) 

(10) 


where  the  ij- th  element  of  A  and  B  are  given  by 

nx 

Aij  = 

t=p+ 1 

and 

ny 

Bij  =  y  ^  Ut—iVt—j  ? 

t=p+ 1 

and  the  i-th  entry  in  a  and  b  axe  given  by 


nx 

O'i  ~  ^  ^  XfXt—i 

t=p+l 

and 


ny 

i>i  =  ^2  ytyt-i  ■ 

t=p+ 1 

Using  the  solutions  of  equations  (9)  and  (10),  denoted  as  a  and  /3,  estimates  of  and 
are  obtained  from 


nx  p 

o-l  =  (i/(«*  - p))  ( Xt  ~'52&k  xt~k )2  (u> 

t=p+l  fc=l 

and 

Uy  P 

dj  =  (i/(ny  -p))  (yt-^2h  yt-kf  ■  (12) 

t=p+ 1  fc=l 
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The  maximum  of  the  numerator  is  obtained  at  the  value  of  a(/3)  that  satisfies 

Coc  =  c  ,  (13) 

where 

nx  ny 

Qj  =  ^  ^  yt-iyt-j 

t=p+ 1  t  =p  + 1 

and 


nx  ny 

Ci  =  ****-<+  ytyt-i  ■ 

t=p+ i  t=p+i 


The  estimate  of  <r2  is 


«®  p  nvp 

^2  (xt  -  £  (yt  -  £  &kyt-k)2 

t=p+i  k~l  t=p+ l  fc=1 


/(nx  +  ny  -  2p)  , 


(14) 


where  a  is  the  solution  of  equation  (13). 

Substituting  &,  3,  cr2,  <t2,  a,  and  <r2  into  equations  (2)  and  (5)  yields 

(^.2)-(m+ny-2p)/2 

_  (^)-(--p)/2(a2)-(ny-P)/2  ’  ^ 

Talcing  —2  •  log  T  of  equation  (15)  yields 

-  2  •  log  T  =  (nx  +  ny  -  2p)  log  <r2  -  (nx  -  p)  log  b\  -  (ny  -  p)  log  a\  .  (16) 

Under  the  null  hypothesis,  tx  =  /3  and  <x2  =  <r2,  -2  •  log  T  has  an  asymptotic  chi-square 
distribution  with  p  +  1  degrees  of  freedom. 
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2.2  TEST  FOR  EQUAL  AR  PARAMETERS,  PROCESS  MEANS,  AND 
INNOVATIONS  VARIANCES 

The  previous  section  derived  the  test  statistic  for  zero-mean  processes.  In  this  section, 
the  test  is  extended  to  include  the  case  where  xt  and  yt  have  means  fix  and  fiy,  respectively. 
Including  the  means  in  equation  (1)  results  in 

p 

Xt  =  fXx  “l-  ^  ]  Oik  —  k  t^x  )  "h  j 
k=l 


V 

yt  =  +  &(  yt-k-yy)  +  vt . 

k= 1 


(17) 


If  equation  (17)  is  used,  expression  (2)  becomes 


L(x, y;  a,  /3,  crx  ,  cry, \ix,  fj,y)  =  (27r^)-(nx-P)/2  (27r<72)-(ny-P)/2 

™  f  v  \2 

-  V  /2< 

1  v  *=!  / 


•exp 


-exp 


Bv  /  p  \ 2 

-  ( (& -/*»)-£ (yt-k - tiy) )  /2<r2 

l  A  / 


(18) 


and  the  test  statistic  is  equation  (6). 

The  mayimum  of  the  denominator  in  equation  (6)  is  obtained  by  solving  equations  (9) 
and  (10),  where  the  ij- th  element  of  A  and  B  are  given  by 

nx 

Aj  ~ 

t=p+l 

and 


ny 

Bn  =  yt-iVt-j , 
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and  the  z-th  entry  of  a  and  b  axe  given  by 


nx 

0,{  ~  ^  %t—i 

t=p+l 


and 


bi  =  ytVt-i  ■ 

t=p+ 1 

In  these  expressions,  xt  =  xt  -  x  and  yt  =  yt  -  y,  where  x  =  Y%=i  xt  / nx  and 

y  =  YZhytlny- 

As  in  equations  (11)  and  (12),  the  estimates  of  a\  and  cr*  are  given  by 


p 

a\  =  (1/ {nx  -  p))  T(xt-t  &k  xt-kf  (19) 

t=p+i  fc=1 

and 

ny  p 

=  (1  /(ny-p))  ^2(yt-J2hyt-k)2-  (20) 

t=p+i  k~1 


The  maximum  of  the  numerator  is  obtained  by  solving  equation  (13),  where  the  ij- th  entry 
of  C  is 


'IJ 


=  I]  *t- 

t=p+l 


ny 

iXt-j  “1“  ^  ^  Vt—iVt—j  5 
t-p+l 


and  the  i- th  entry  of  c  is 


nx  ny 

d  =  E  +  E  v*ih-i  • 

t=p+i  t=p+ 1 


In  these  expressions,  xt  =  xt  —  ft  and  yt  =  yt  —  fa  where  /i  =  ( nxx  +  nyy)/(nx  +  ny). 
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Equation  (14)  now  becomes 


a2 


p 

Xt  ®k%t—k 

k=  1 


/  ( nx  +  ny  —  2 p) . 


(21) 


The  test  statistic  of  equation  (16)  can  be  evaluated  from  equations  (19),  (20),  and  (21). 
Under  the  null  hypothesis  a  =  (3,  <j\  =  cr2,  px  =  fiy,  and  expression  (16)  has  a  chi-square 
distribution  with  p  +  2  degrees  of  freedom. 


2.3  TEST  FOR  EQUAL  AR  PARAMETERS 

In  many  applications,  there  is  interest  only  in  the  AR  structure  of  the  process,  not  in  the 
means  {px  and  fxy)  and  the  innovations  variances  (cr2  and  cr2). 

The  denominator  of  test  statistic  (7)  is  maximized  by  using  equations  (9)-(12).  However, 
the  numerator  is  maximized  by  solving  for  a,  cr2 ,  and  cr2  in  the  implicit  system  of  equations: 

Da  =  d  ,  (22) 


/  p  \2 

<r2  =  1/ (nx  —  p)  y"'  (  xt  -  £  ®kXt-k  )  , 

1  V  *=1  J 


ny  /  p  \  * 

=  1  /(ny-p)  Y  ( yt  -  £  ®kyt-k  )  , 
i  V  fc=i  / 

where  the  ij- th  entry  of  D  is 


ny 


Dij  =  1/cr2  Y  Xt-iXt-j  +  l/cr2  Y  Vt-iVt-j 

t=p+l  t=p+ 1 


and  the  i-th  entry  of  d  is 


nx 


ny 


di  =  1  /cr2  Y  xtxt-j  +  1/al  Y  ytVt~i 

t=p+ 1  t=p+i 


(23) 


(24) 
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These  equations  may  be  solved  using  a  fixed-point  algorithm.  The  procedure  begins  by 
estimating  a  using  equation  (22);  values  for  a\  and  a\  axe  those  obtained  from  maximizing 
the  denominator  of  equation  (7).  With  the  estimate  for  a,  new  estimates  are  obtained  for 
a\  and  al  from  equations  (23)  and  (24).  This  procedure  is  repeated  until  the  change  in 
value  of  the  likelihood  function  is  less  than  a  user-selected  convergence  criteria.  Typically, 
the  algorithm  converges  in  less  than  five  iterations. 

The  complete  likelihood  function  must  be  used  to  evaluate  T  because  the  terms  involving 
the  exponential  do  not  cancel.  Taking  —2  •  logT  yields 


-2-logT  =  {nx  -  p)  log  a2u  +  {ny  -  p)  log  a\ 


nx  s  p  W  ny  ,  p  n; 

+(i/*J)  E  ( **  -  £  QkXt-k )  +  (i /si)  (yt-Y,  otyt-k ) 

V  k= 1  )  v  *==1  / 

-{nx  -  p)  log  al  -  {ny  -  p)  log  a\  -  {nx  -  p)  -  {ny  -  p) 


(25) 


or 


-2  •  log  T  =  {nx  —  p)  log  {al/ cl)  +  {ny  -  p)  log  (oj/, a) 

nx  ,  p  \  2  ny  ,  p  .2 

+(i/«i)  E  (**-£ +  (1/5J)  E  (jft-E m-k 

— (na:  +  ny  -  2p),  (26) 


where  a,  al,  a\  maximize  the  numerator  and  a,  ^3,  <3^  maximize  the  denominator  of 

equation  (7),  respectively.  As  before,  —2  •  logT  has  a  chi-square  distribution,  but  with  p 
degrees  of  freedom. 


2.4  TEST  FOR  EQUAL  AR  PARAMETERS  AND  PROCESS  MEANS 

For  completeness,  the  case  for  equal  AR  parameters  and  process  means  is  examined. 
Now  the  denominator  of  test  (8)  is  maximized  by  using  equations  (9)  through  (12), 
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where  xt  and  yt  are  replaced  by  xt  -  x  and  yt  -  y,  respectively  ( x  =  (1  fnx)  Y%L  1  xt  and 
y  =  (1  jny)  Vt)-  The  numerator  is  maximized  by  equations  (22)  through  (24),  where  xt 

and  yt  are  replaced  by  xt  -  ft  and  yt  -  £,  respectively  {fi  =  (nx  ■  x  +  ny  -y)/ (nx  +  ny)). 
The  expression  for  the  test  statistic  is  given  by  equation  (26).  This  value,  —2  •  log  T,  has  a 
chi-square  distribution  with  p  +  1  degrees  of  freedom  if  a  =  (3  and  px  =  py. 
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3.  LIKELIHOOD  RATIO  TESTS  FOR  ARMA  PROCESSES 


This  section  extends  the  previous  results  to  ARMA  processes.  Let  these  processes  xt  and 
yt  be  defined  by 


p  ? 

xt  =  fix  +  ak  ■  ( xt-k  ~  Vx)  +  Ut  +  Yl  OkUt-k 

k= 1  fc=l 


(27) 


and 


yt 


p  q 

Vy  +  ^Pk'  ( Vt-k  ~  liy)  +  1H  +  faVt-k  , 
fc=l  fc=l 


(28) 


where  ut  and  vt  are  defined  as  in  equation  (1).  As  for  the  AR  models,  the  ARMA  model 
orders  p  and  q  are  the  same  for  both  processes  xt  and  yt . 

The  Gaussian  likelihood  of  xt  and  yt,  written  in  terms  of  conditional  probabilities,  is 


ny 


L(x,  y;  a,  9,  / 3 ,  <f>,  <x2 ,  er2,  ptx,fiy)  =  JJ  a,  9,  <r2,  fix)  JJ  fy(yt |y*-i,  (3,  <f>,  cr2y,  fj,y), 

(29) 


1 


t= 1 


where 


a  =  (aia2,...,ap)T  , 

9  =  (M 2,...,0qf, 

P  -  (/31,/52,...,/3p)T, 
4>  =  ) 

X.t—1  =  (%!■>  X2)  •••,  Xt— l)  , 


yt-i  =  (2/x,2/2,-,yt-i)T  > 


/x(^l|xO)  ot,  9, cr2,  y,x) 

fy(vi\yo,0,4>,ol,i*y) 

fx(xt\xt-i,a,9,al,iJix) 
fy{yt\yt-i,P,<t>,<7l,yy) 
x(t\t  —  1) 
y(t\t  - 1) 


fx{xi\ot,9,a2, px)  , 
fv(Vl\0,<f>,ol>l*y)  > 

(27r<T^Ot)_1/2exp(-(a:t  -  (jjlx  +  x(t\t  -  l)))2/2cr2at)  , 
(2t r<T26t)“1/2  exp(-(yt  -  (/*„  +  y(t\t  -  l)))2/2a26t)  , 
a,  9,o\,  px]  , 

E[yt  -  fj,y\yt- i,/3,^,a2,)Uj,]  , 
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a2uat  =  Var[xt\Stt-i,a,0,a2u,nx]  , 

(7%  =  Var[yt\yt-i,(3,4>,al,ny]  . 

Var[- ]  denotes  the  variance  of  the  random  variable  in  the  brackets.  Substituting 
f2.(-)  and  fj,(-)  in  equation  (29)  and  taking  the  natural  logarithm  yields  the  log  likelihood 
of  xt  and  yt.  When  the  constant  terms  involving  2tv  are  ignored,  the  log  likelihood  is  given  by 

-2  •  log  L(x,  y;  a,  9,  (3,  <f>,  a2u,  <r2v,  fix,  ny)  = 


-nx  •  log  a2u  -  log  at  -  Y^(xt  -  (nx  +  x(t\t  -  1  )))2/<r2u  at 


t=  1 


i=l 


ny  ny 

-ny  ■  log  a2v  -  log  bt  -  ]T( yt  -  (yy  +  y(t\t  -  1  )))2/<rJ  bt 


t=  1 


t=l 


(30) 


The  log  likelihood  can  be  evaluated  with  a  Kalman  filter  (see  appendix)  algorithm 
and  can  be  maximized  with  respect  to  a,  0,  /3,  4>,  a2,  a2,  nx,  and  yy  via  a  nonlinear 
optimization  procedure.  The  quantities  at  and  bt  are  calculated  as  part  of  the  Kalman  filter 
evaluation  of  the  log  likelihood.  As  in  section  2,  four  likelihood  ratio  tests  are  presented  for 
the  equivalence  of  ARMA  processes.  The  only  difference  from  the  AR  tests  is  the  inclusion 
of  the  moving-average  parameters. 


3.1  TEST  FOR  EQUAL  ARMA  PARAMETERS  AND 
INNOVATIONS  VARIANCES 

The  test  is  for  a  =  f3,  0  =  (f>,  and  <j2u  =  a2v.  As  in  the  AR  test,  the  sample  means  x  and 
y  are  removed  from  xt  and  yt,  respectively.  For  this  case,  the  likelihood  ratio  test  is 

max  L(x,  y;  a,  0 ,  /3,  <\>,a2u,  a2v) 
a  =  (3,0  =  <(>,a2u  =  a2v 

T  =  —  ~  —  2  2\  '  (^1) 

max  L(x,  y;  a,  9, (3,  b,a ;,  a;) 

a,  0,(3,  <t>,<r2u,a2v 

Taking  -2  •  log  T  yields 
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(32) 


nx  ny 

-2  •  log  T  =  (nx  +  ny)  •  log  a2  4-  log  Of  +  ^  log  bt 

t=i  t=i 

nx  ny 

-nx  •  log  &l  -  ^  log  at-ny-  log  a2y  -  ^log  bt  , 

t=l  t=  1 

where  a2,  at,  and  bt  axe  obtained  by  maximizing  the  numerator  of  equation  (31)  and  a2, 
a2,  at,  and  bt  are  obtained  by  maximizing  its  denominator.  For  this  case,  —2  •  log  T  has  an 
asymptotic  chi-square  distribution  with  p  +  q  +  1  degrees  of  freedom. 


3.2  TEST  FOR  EQUAL  ARMA  PARAMETERS,  INNOVATIONS 
VARIANCES,  AND  PROCESS  MEANS 

The  test  in  equation  (32)  assumed  zero  mean  or  equal  mean  data.  Here,  the  test  for 
fj,x  —  p,y  is  included  in  the  likelihood  ratio  test: 

max  L(x,  y;  a,  6,  (3,  <p,a2u,  c\,  px,py) 
a  =  (3,0  =  <t>,al  =  al,  px  =  py 

T  =  - 77 - sm  Z . 2-  2 - (33) 

max  L(x,  y;  a,  0,  /3,  <J%,  px,  py) 

a,  0,  /3,  <t>,  a\,  /xa,  ny 

Taking  —2  •  log  T  yields  equation  (32),  where  a2,  5*,  and  bt  are  obtained  by  maximizing 
the  numerator  of  equation  (33)  and  a2,  a2,  (it,  and  bt  are  obtained  by  maximizing  its 
denominator.  Here,  —2  •  log  T  has  an  asymptotic  chi-square  distribution  with  p  +  q  +  2 
degrees  of  freedom. 


3.3  TEST  FOR  EQUAL  ARMA  PARAMETERS 

In  many  applications,  only  the  ARMA  structure  of  the  processes  is  of  interest.  The  test 
for  a  =  (3  and  0  —  <f>  is  given  by 

max  L(x,  y;  a,  0, (3,  <t>,<rl,  o\,  px, / iy ) 
ct  =  (3,0  =  <t>,a2u,  a2v,  px,  py 

T  =  - .  (34) 

max  L(x,  y;  a,  0,  (3,  <f>,a2u ,  a\,  px,  ny) 

a,  0,  (3,  <f),a-2u,  a\,  px,  fxy 
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The  sample  means  x  and  y  are  subtracted  from  the  data  x  and  y,  respectively. 
Substituting  equation  (29)  into  the  numerator  and  denominator  of  equation  (34)  and  then 
maximizing  the  numerator  and  denominator  yields  the  test  value  of  the  statistic  T.  Taking 
-2  •  log  T  yields 


nx 

-2  •  log  T  =  nx-  log  d\  +  ^  log  at 

t-i 

ny 

+ny  -  log  o-2y  +  y>g  bt 

t= 1 
nx 

-nx  -  log  d\  -  ^  log  dt 

t=  1 
ny 

-ny  -  log  v2y-^>2  lo§  ht  ’  (35) 

t=l 

where  dx,  d2,  dt,  and  bt  are  obtained  from  maximizing  the  numerator  and  d2,  d2,  dt,  and  bt 
are  obtained  from  maximizing  the  denominator.  For  this  case,  —2  •  log  T  is  asymptotically 
chi-square  distributed  with  p  +  q  degrees  of  freedom. 


3.4  TEST  FOR  EQUAL  ARMA  PARAMETERS  AND  PROCESS  MEANS 

In  this  special  case,  there  is  a  trend  in  the  data.  The  likelihood  ratio  test  for  a  =  (3, 

0  =  <p ,  and  px  =  py  is 

max  L(x  -ffi,y-fh;cx,  0,  (3,  (j>,a2u,  a\,  px,  py) 
cx  =  P,0  =  4>,<rl ,  cr2v,  px  =  py 

fjp  ——  _  (36^ 

max  L(x  —  x,y  —  y-,a,  0,  P,  4>,o\,  a2,  px,  py)  ’ 

Pi  &i&ui  ®v>  l^xi  f^y 

where  x  =  (1  /nx)  YZi  V  =  0-/nV)  E?=i  and  m  =  (1  /(nx  +  ny))  +  YT  Vt]  ■ 

Maximizing  the  numerator  and  denominator  with  equation  (29)  and  then  taking  —2  •  log  T 
yields 


nx 

—2  •  log  T  =  nx-  log  d\  +  log  at 

t=i 
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(37) 


ny 

+ny  •  log  fr\  +  £>g&t 

t=l 

mr 

-nx  ■  log  fr\  -  log  a* 

t=l 

n2/ 

-ny  •  log  dj  -  J]log  bt  , 

t= 1 


where  5^,  5^,  at,  and  bt  are  obtained  by  maximizing  the  numerator  of  equation  (36) 

and  6-3,  fry,  at,  and  bt  are  obtained  by  maximizing  its  denominator.  Here  -2  •  log  T  is 
asymptotically  chi-square  distributed  with  p  +  q  +  1  degrees  of  freedom. 
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4.  MONTE  CARLO  ANALYSIS 


In  this  section,  the  time  series  test  is  examined  to  see  how  well  it  fits  the  chi-square 
distribution.  As  an  example,  the  test  of  equation  (16)  is  applied  to  two  AR(4)  time  series, 
both  defined  by 


xt  =  2.0625a:t_i  -  2.4325xt_2  +  1.5847zt_3  -  0.652rrt_4  +  Ut  .  (38) 

Two  independent  realizations  of  the  above  process  were  simulated.  The  AR  model 
order  was  chosen  using  the  AIC  selection  procedure.  The  value  of  the  model  order,  p, 
used  in  the  test  was  selected  from  the  pooled  process.  The  test  statistic  was  calculated  for 
10,000  pairs  of  realizations  with  a  sample  size  of  200.  This  statistic  was  then  compared 
against  a  chi-square  distribution  with  (p  +  1)  degrees  of  freedom.  As  a  function  of  model 
order,  table  1  shows  the  fraction  of  the  test  statistic  values  that  exceeded  the  chi-square 
percentage  points  for  the  significance  level  indicated  in  the  top  row.  These  results  indicate 
that  the  test  has  a  good  fit  to  the  chi-square  distribution.  However,  the  test  is,  in  general, 
conservative,  which  means  that  the  test  rejects  the  null  hypothesis  less  often  than  predicted 
by  the  chi-square  distribution.  Depending  on  the  application,  a  conservative  test  may  be 
more  desirable  than  a  liberal  test  because  it  has  fewer  false  alarms. 


Table  1.  Fraction  of  the  Test  Statistic  Values  Exceeding  the  Chi-Square 
Percentage  Point  for  Equal  Models 


Order 

HlBEH 

0.25 

0.1 

msm 

0.025 

0.01 

4 

5642 

0.4975 

0.2407 

0.0939 

0.0473 

0.0245 

0.0105 

5 

1559 

0.4894 

0.2450 

0.0943 

0.0500 

0.0218 

0.0051 

6 

1033 

0.5073 

0.2536 

0.1162 

0.0610 

0.0358 

0.0181 

7 

845 

0.4852 

0.2379 

0.0959 

0.0367 

0.0107 

0.0047 

8 

921 

0.5005 

0.2617 

0.0966 

0.0445 

0.0206 

0.0087 

combined 

10000 

0.4965 

0.2444 

0.9670 

0.0480 

The  previous  paragraph  analyzes  the  test  statistic  when  the  null  hypothesis  is  true.  Now 
the  power  of  the  test  is  examined  when  the  AR  processes  are  different.  One  of  the  processes 
is  as  defined  above  for  xt\  the  second  is  defined  by 

yt  =  1.971yt_i  -  2.339yt_2  +  1.5208yt_3  -  0.6544yt_4  +  vt  . 
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The  simulation  procedure  described  above  was  applied  to  these  two  processes,  with  the 
results  presented  in  table  2.  At  the  0.05-significance  level,  the  test  rejects  the  equality 
of  the  model  parameters  of  the  two  processes  for  15  to  20  percent  of  the  realizations,  as 
compared  to  5  percent  when  the  two  processes  axe  the  same.  These  results  are  encouraging, 
considering  the  “closeness”  of  the  two  processes,  which  is  measured  by  the  location  of  the 
zeros  for  the  characteristic  equation  of  the  AR  process.  Table  3  shows  the  characteristic 
equation  zeros  for  the  two  processes.  A  larger  separation  between  the  zeros  than  shown  in 
this  table  will  result  in  a  greater  rejection  of  equality. 


Table  2.  Fraction  of  the  Test  Statistic  Values  Exceeding  the  Chi-Square 
Percentage  Point  for  Unequal  Models 


Order 

Samples 

0.5 

0.25 

0.1 

0.01 

4 

0.7483 

0.5256 

0.0634 

5 

0.7366 

0.5115 

0.3015 

0.1915 

0.1126 

6 

1024 

0.7119 

0.4932 

0.2734 

0.1133 

7 

868 

0.7051 

0.4493 

0.2385 

0.1336 

0.0680 

0.0323 

8 

998 

0.7074 

0.4689 

0.2485 

0.1483 

0.0882 

0.0451 

combined 

10000 

0.7349 

0.5078 

0.2895 

0.1831 

0.1105 

0.0593 

Table  3.  The  Coefficients  and  Zeros  of  the  Two  AR(4)  Models 


Series  1 

Series  2 

Coefficients 

2.0624611 

.  1.9710257 

-2.43250 

-2.3390543 

1.5846875 

1.5208008 

-0.6520562 

-0.6543516 

Zeros  (polar) 

(0.950  ,  ±0.100) 

(0.945  ,  ±0.105) 

(0.850  ,  ±0.200) 

(0.856  ,  ±0.205) 
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5.  CONCLUSIONS  AND  RECOMMENDATIONS 


The  analysis  in  this  report  has  studied  the  equality  of  the  parameters  for  two  ARMA 
processes.  Four  likelihood  ratio  tests  were  derived  to  compare  (a)  the  ARMA  parameters 
and  innovations  variances,  (b)  the  ARMA  parameters,  innovations  variances,  and  process 
means,  (c)  the  ARMA  parameters,  and  (d)  the  ARMA  parameters  and  process  means. 
The  likelihood  ratio  test  statistic  has  been  shown  to  have  a  good  fit  to  the  chi-square 
distribution  when  the  models  are  identical. 

The  test  statistics  presented  here  are  examples  of  the  generalized  likelihood  ratio  test 

maxL(x,  y;  0X,  6y) 

rp  _  As  =  6y _ 

max  L(x,  y;  9X,  6y ) 

&x  j  9y 

where  L(-)  is  the  likelihood  of  the  data  and  0X  and  6y  are  the  p-vectors  of  the  model 
parameters  that  characterize  the  data.  As  shown  in  the  report,  —  2  •  logT  has  an  asymptotic 
chi-square  distribution  with  p  degrees  of  freedom  when  the  null  hypothesis  (0X  =  9y)  is 
true. 

Three  recommendations  for  further  research  are  (1)  to  extend  the  test  to  complex-valued 
processes,  (2)  to  study  the  sensitivity  of  the  power  of  the  test  when  the  two  process  models 
become  close,  as  measured  by  the  zeros  of  the  characteristic  equation,  and  (3)  to  expand 
the  test  for  three  or  more  time  series. 

For  future  sonar  systems,  these  test  procedures  can  be  utilized  in  the  automatic 
detection  and  classification  of  sonar  signals,  which  will  reduce  the  workload  and  improve 
the  performance  of  the  sonar  operator. 
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APPENDIX 

LOG  LIKELIHOOD  EVALUATION  FOR  THE  ARMA  MODEL 

This  appendix  presents  a  Kalman  filter4-6  procedure  for  evaluating  the  log  likelihood 
of  L(x;  a,  0,  a2u,  /i*)  (or  log  L(y,  0, 0,  o*,  /x ,)),  given  the  model  parameters.  For  the  Kalman 
filter,  let  the  observation  xt  be  given  by  the  observation  equation 

xt  —  zTwt  +  rt  ,  (A-l) 

where  xt  is  the  observed  data,  z  is  an  m  x  1  vector  of  fixed  known  values,  wt  is  the  state 
vector  of  the  system,  and  rt  is  the  measurement  error.- 

The  state  equation  of  the  system  is  given  by 

Wt  =  Awt-i  +  Bet  > 

where  A  is  an  m  x  m  transition  matrix  of  fixed  known  values,  B  is  an  m  x  fc  matrix  of  fixed 
known  values,  and  et  is  a  k  x  1  vector  of  Gaussian  random  variables  with 

E[et]  =  0, 

E[etef]  =  a2Q  , 

E[eteTs)  =  0  for  t  ^  s  . 

The  known  matrix  Q  is  positive  definite.  Further, 

E[rt]  =  0, 

E[r2\  =  R , 

E[rtrs 1  =  0  for  $  ±  t  , 

E[rtes]  =  0  for  all  s  and  t . 

Let  w(t  -  1  |  t  -  1)  denote  the  minimum  mean-square  estimate  of  wt-i  given  the 
measurements  xu  x2, ...  ,  xt-\.  Let  a2P(t- 1  \t-l)  denote  the  estimation  error  covariance 
matrix  where  P(t  -  1  1 1  -  1)  is  known.  That  is, 

£[(«,,-,  -  w(t  -  1 1 1  -  l))(w,_i  -  w(t  —  1 1 1  —  1))T]  =  tr2P(£  -  1 1 1  -  1)  . 


A-l 


Let  w(t  1 t  —  1)  and  P(t  1 1  —  1)  denote  the  predicted  values  of  wt  and  P(t  1 1)  given  x\, 


X2,  ...  ,  xt~\.  These  quantities  are  given  by  the  prediction  equations 

w(t  |  t  —  1)  =  Aw(t  —  1  1 1  —  1),  (A-2) 

P(t  1 1  -  1)  =  AP(t  -  1 1 1-  1  )At  +  BQBt.  (A-3) 

Using  the  observation  xt,  w(t  \  t)  and  P(t  1 1)  are  obtained  by  the  update  equations 

w(t  |  t)  =  w{t  1 1  -  1)  A  P{t  1 1  -  l)zatl(xt  -  zTw(t  1 1  -  1))  ,  (A-4) 

P(t  |  t)  =  P(t  1 1  -  1)  -  P{t  1 1  -  1  )zaJlzTP(t  1 1  -  1)  ,  (A-5) 


where  at  =  zTP(t  1 1  —  1  )z  +  R. 

The  ARMA(p,?)  model  for  xt,  given  in  equation  (27),  can  be  written  as 

r  r 

xt  =  ^2  akxt-k  +  ^2  °kut-k  +  Ut  ,  (A-6) 

fc=l  k= l 

where  r  =  max(p,§  +  1).  Alternatively,  equation  (A-6)  can  be  expressed  in  Markovian  form 
by 


ai 

1 

0  . 

. 

0  " 

'  1  • 

0.2 

0 

1  . 

. 

0 

• 

. 

. 

02 

Wt  = 

• 

• 

• 

Wt- 1  + 

• 

a-r- 1 

0 

.  0 

1 

.  OLr 

0 

. 

.  0 

0  . 

.  0r-l  . 

where  the  first  element  of  wt  is  xt.  Equation  (A-7)  is  the  state  equation  in  the 
state  space  formulation  of  the  ARMA(p,<?)  model.  The  measurement  equation  is  xt  =  zTwt, 
where  2  =  (1,  0,  ...  ,0r)T. 

For  the  ARMA(p,g)  model  without  measurement  error,  R  =  0,  Q  =  1, 
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Oi\  1  0  ...  0 

<*2  0  1  ...  0 


ar_i  0  ...  0  1 

aT  0  ...  0  0 

and 

’  1 

02 

B  = 

.  Or- 1 

To  start  the  recursions  in  equations  (A-2)  through  (A-5),  initial  values  are  needed  for 
w(0  |  0)  and  P(0  |  0).  Without  any  measurements,  the  minimum  mean-square  estimate  of 
wo  is  zero;  hence,  tu(0  |  0)  is  set  to  zero.  Now,  since  P( 0  |  0)  =  [wqWq] ,  the  covariance 
matrix  of  the  state  vector,  the  value  of  P( 0  |  0)  is  the  solution  P  of  the  equation 

P  =  APAt  +  BQBt  .  (A-8) 

Using  equation  (A-2)  and  noting  the  stationarity  assumptions  on  the  model  allows  equation 
(A-8)  to  be  expressed  as 

[I  —  A  <S>  A]vec(P)  =  vec(BQBT ),  (A-9) 

where  <S>  is  the  Kronecker  product.  To  solve  this  system  of  r2  equations,  the  r2  x  r2  matrix 
[I  —  A  <S>  A]  does  not  require  inversion.  Rather,  the  columns  of  P  can  solved  recursively 
using  the  following  relations: 

pi  =  (;-em‘)  '(e/V!*).b  (a-io) 

and 
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Pi  =  otiApi  +  Api+i  +  Pi^B  for  i  >  1  . 


(A-ll) 


After  solving  for  P$  =  P,  the  log  likelihood  can  be  evaluated  using  equations  (A-2)  through 
(A-5),  given  the  ABMA(p,q)  model  parameters. 
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